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Abstract 

In this paper we prove a Multivariate Laplace approximation with estimated 
error for two cases of maximum, in the interior of the domain and on the boundary. 
Additionally, the function in the exponent and its point of maximum depends on 
the integrals asymptotic parameter. As an application, we prove Law of Large 
Numbers and the Central Limit Theorem for the random vector related to the 
Laplace Integral. The second result gives different limiting distribution for two 
cases of maximum in the Laplace integral. When it is in the interior of the domain 
it is Gaussian distribution and on the boundary it is exponential in one direction 
and Gaussian in other directions. 

Key words — Multivariate Laplace method, Laplace method. Asymptotic approxi¬ 
mation of integrals. Law of Large Numbers, Central Limit Theorem 


1 Introduction 


We prove Multivariate Laplace approxi mation which is an ext e nsion of results already 


known in the literature, see for example Wong |2nni |, Breitnng 19941. Additionally, we 


consider the function in the exponent of the integral and its point of maximum depen¬ 
dent on ’large’ parameter. We also provide rate of convergence and estimate of the 
re mainder term of d eveloped approximations. These results are development of results 


Kolokoltsov j2nnn |. Moreover, we consider two cases of maximum, in the interior and 


m 

on the boundary of the domain. 

For the probability application we hrst dehne a probability space, with domain of 
Laplace integral as a set and Borel a-algebra on it. The distribution of particular event is 
constructed by taking the Laplace integral over the event set and normalizing by integral 
over the domain. We consider a random vector whose values are the points of the domain 
of considered integral. The hrst probabilistic result. Law of Large Numbers gives us mean 
equal to the point of maximum of the function in the exponent of Laplace integral. The 
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second result, Central Limit Theorem gives us the distribution of fluctuations different for 
two cases of maximum. When it is in the interior of the domain it is Gaussian and on the 
boundary, exponential in one direction and Gaussian in other directions. Additionally, 
rate of convergence for both limit theorems is included. In the proof we used developed 
Laplace approximation and some rather standard estimates. 

The paper consists of three sections and appendix. Second section contains our main 
results. Multidimensional Laplace approximation. First we consider one dimensional case 
with maximum on the boundary. Then we consider multidimensional case with the max¬ 
imum in the interior and then general case on the boundary. In the third section we 
develop the probability application. We introduce the probability space and random vec¬ 
tors, state and prove two limit theorems. In the end of this section we include additional 
estimates used in the proofs of the probability results. In the appendix we included some 
standard, easily obtained results used throughout the paper. 


2 Laplace approximation 

We consider an integral over a bounded domain hi C 

I{N) ■= f (1) 

Jn 

where N > Nq, for some No G N, are positive integers. 

Function / ; hi x N —)■ M converges to M-valued function as N ^ oo and have a unique 
maximum in the points denoted x*{N) for all N > Nq and x* in the limit. Additionally, 
it is three times continuously differentiable in some closed neighborhood hi' of x* and 
those derivatives are uniformly convergent as N ^ oo. 

Further, we consider two types of maximum of /: 

(a) in the interior of hi and x* is a non-degenerate critical point, 

(b) on the boundary dQ where x* is a non-degenerate critical point and non-critical in 
the orthogonal direction. Here we assume dQ is a hyperplane in some neighborhood 
of X*. 


Function H —)■ M is continuously differentiable in and bounded in H. 
From assumptions about / and g and for appropriately chosen we have 


_p(2) , 

:= sup |. 


x£Q',N>No 

_P'(2) 

:= inf 1 


x&n',N>No 

ri'(2) 

Xn 

:= min | 

_p(3) . 

V 

:= sup |. 


x£Q',N>No 

G := 

sup ||^(a;)|| < 




:= sup \\Dg{x 


f{x*{N), N) - sup^g^\^, f{x, N) 
N>No sup,,gf 2 \f 2 ' \x-x*{N)\'^ 


> 0 , 


ceo' 


( 2 ) 

(3) 

(4) 

(5) 

( 6 ) 
(7) 


2 



Additionally, we choose hi' and A"o snch that for all N > Nq the points x*{N) G hi' and 
for all X ^ fl', 

X<\det{D^f{x,N))\<A, (8) 

where A and A are some positive constants. 

For the maximnm of type {b) the derivatives in above determinant and those in the 
constant are w.r.t. coordinates of the hyperplane dQ at x* and also only for this 
maximnm we define 


:= inf |/'(a:,A^)| > 0, 
x&n',N>No' ^ 


:= min F'^^\ inf 
“ ' N>No 


f{x*{N), N) - sup^gf^\f^, /(x, N) 


sup^eo\n' k-x*(iV)| 
where the derivative is in direction orthogonal to that hyperplane. 


> 0 , 


(9) 

( 10 ) 


2.1 One-dimensional domain with the maximum on the bound¬ 
ary 

Theorem 1. For the integral ([7]) with hi C M and the maximum of type (b), we have 
following approximation 



Nf(x*,N)}_ 

N 


\nx*,N)\ 


+ 


^b{N)\ 

N J’ 


where ub{N) is some bounded function and 


|a;B(iV)| < 




+ 


G \rl/2p^(l) 

_Ne-^ + 


y(i) ■ 


F'W ^ 


Proof. Let ns define 




N\f'{xpN)\ 




and introdnce the set Un := {x : |x — x*| < 1/iV^/^, N > A"o}, with Aq large enongh 
that Un C hi'. Then by separating the set Un and with use of Taylor’s Theorem we 
decompose I{N) and Ig{,N) 


I{N) =In{N) + h2{N) + h{N) 



g{x*) ( e^^^^'^^dxF f g'{xei){x — x*)e^^^^’^^dx+ 
Jun JUn 


I^{N) = Igi{N) + Ig 2 {N) := g{x*)e^f^^*’^^ [ 

Jun 

+ f e-^\ri-*F)\ix-*)dx. 

Jn\UN 

Now, we put together above decompositions 


|J(iV) - Ig{N)\ < |/n(iV) - /Gi(iV)| + |Ji2(iV)| + |/2(iV)| + |/G2(iV)|, 
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and approximate each term. 

To approximate the expression |Jii(iV) — we apply Taylor’s theorem for / and 

inequality |e* —1| < Further, without loss of generality we can assume f'{x*, N) < 0 

and get 


|/n(iV)-/Gi(iV)| = 




< 


< 


.N 




IUn 


Since the integration is over Un, have that \x — x*\ < hence above expression is 
bounded by 


IUn 


and the integral 


\x 




JUn 

Hence we have 


(iV|/'(x*,iV)|)3- 


\IniN) - IoAN)\ < 

For the integral |/i 2 (iV)| we use Taylor’s Theorem f{x, N) = f{x*, N) + f'{x 0 ^{N), N){x — 
X*) to get an estimate for /. Since f{x*, N) is its uniqe maximum and Un C ff' we get 

f{x,N) < f{x\N) -F'^^^\x-x*\, 

where is given by ([9]). 

We insert it into 1 12 and as g has bounded derivative in [/at we integrate 

poo 1 

|/i 2 (iV)| < / g'{xe,){x - 

A* (iVF'd)) 

For |/ 2 (A^)| we substitute the estimate of / as for the integral I 12 , but here we have 
possibly smaller constant dehned by flTU]) . Then hnd an upper bound and integrate 




1 Ari/2_p'(i) 


NK 


'( 1 ) 


n 


The integral \Ig 2 {N)\ we approximate similarly, but instead of xo^{N) we have x* in the 
hrst derivative of Taylor expansion, hence 

1 






N\r{x%N)\ 

Now, we combine all approximations and conclude with 

1 


|/(iV) -/G(Ar)| < 




[ (F'(b) 


9 S- 77T\^ ^ 

(F'mf F^) 


|5'(^*)I Tyf -ATl/aiT'Cl) 

F'(1) 


and therefore we proved the theorem. 


□ 
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2.2 m-dimensional function with the maximum in the interior 
of the domain 

Theorem 2. For the integral (QP with C MF and the maximum of type (a) we have 
following approximation 


g^x)e^n-F)dx = 


2 n 


g{x*{N)) 


N J y^det D^f{x*{N),N) ^/N /’ 


(- 


where uJi{N) is some bounded function and 


|a;/(A^)| <71 2 


r( 


m+l 


p(3) 

3F'(2) 


ef'‘*|9(i-(iV))|+G<‘> 


+ 


71 2 


r( 


V(2)x 


X 


X 




Ari/3F'(2) 




Proof. Since / has property ([HD we can define 


I^{N) : = g{x*{N))e^f^^*^^^’^^ 


( 277 \ 


\N J ^det D^f{x*{N),N) 

= [ exp {^N{x — x*{N)Y'D"^f{x*{N).,N){x — x*{N)'^dx., 

/lUm / 


and introduce the set = {x ■. \x — a;*(iV)| < N > iVo} with Nq large enough to 

Un (ZQ'. Then by separating the set f/jv and using Taylors theorem we decompose I{N) 
and IciN) 


I{N) =/ii(iV) + Ji 2 (iV) + hiN) := g{x*{N)) / 


'Un 


+ / Dg{xe,{N)f{x - x*{N))e^f^^’^^dx + / g{x)e^f^^’^Ux, 


'Un 


'n\UN 


Ig{N) = Igi{N) + Ig 2 {N) := 


>Un 


= g{x*{N))e^f^^*^^'^’^^ I exp “ x*{N)f f {x*(N), N){x - F 


dx+ 


+ / exp (^N{x-x*{N)fD‘^f{x*{N),N){x-x*{N))]dx 


<Un 


We combine above integrals into 

\I{N) - Ig{N)\ < \In{N) - Igi{N)\ + |Ji2(iV)| + |/2(iV)| + |/G2(iV)|. 

For the expression Iii{N) — Iqi{N) we use third order Taylor’s theorem to obtain 
|Jll(iV)-JGi(iV)| = 

C 7 \^ 

= |(7(a;*(iV))| / exp(Nf{x*{N),N) + -{x-x%N)fD^f{x%N),N){x-x%N)))x 
JUn \ I / 

X (^exp (^^ND^^f{xe,{N),N){x-x*{N))^^'^ -l^dx, 
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where Df{x*{N), N)'^{x — x*{N)) = 0 for all x since x*{N) is a critical point. 

Next, we apply inequality |e* —1| < and use an upper bound \D^^f{xg 2 {N),N){x — 
x*(iV))®^| < F® |a; — a;*(A^)p derived in f[T7|) in Appendix and since \x — a;*(A^)| < 
for X G Fw we conclude 


6 Juj, 

Then using calculus result dy = W^r(^^)/r(Y) we obtain the 

estimate 




JUn 

-r( 

= TT 2 


r( 


m+1 ' 
m\ \ 2 


m+3 

2 


and therefore 


|/ii(A')-/Gi(iV)l < 






For the integral Ji 2 (A^), by Taylor’s Theorem and taking into account that |x—x*|||F^/(xe) ^ 
\D‘^f{x0)'^{x — X*) I we have an upper bound of /(x, N) for any x G 


f{x,N) < f{x*{N),N) - -F'^^\x - x*{N))\ 
and since g has bounded derivative in f/jv we have 

Jun 

and with use of \x\e-^\^\^dy = /T{^) we obtain 


|A2(A^)| < 


r( 


in+i') / pl{2) \ 1 

2 J f ] (^(l)^gW/(x*(V),iV)^-f 




For I 2 {N), since hi is bounded we have the same estimate for / as in J 12 but with constant 


y(2) 


|/2(iV)| < 


"WiV 




and since g is bounded we estimate above using (1T5]) from Appendix 


l^2(iV)| < 


vtt 


r( 


F(m)+ 1 +VF, 




m— 1 - 




V2j 


X 


X 
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In the case of Ig 2 the approximation procedure is analogical to / 2 , yielding 


|/G2(iV)| < 




r(m)+ 1 + VF, 


T(f)V 2 

Then we combine above approximations into 




m—l- 






V 2 j 


X 


\I{N) - Io{N)\ < ' 


-r( 




71 2 


X 




— / 

TT 2 / Ff 




m—l- 


r(f)V 2 


r(f) V 2 

7(2)^ -22^ p 

r(m)+ 


X 


which yields the statement of the theorem. 


□ 


2.3 m-dimensional function with the maximum on the bound¬ 
ary of the domain 

Theorem 3. We consider an integral (QP with hi C M™ and with maximum of type (b). 
If the dfl at the maximum is not orthogonal to first coordinate axis then we obtain it by 
appropriate rotation of coordinate system. Then we have following approximation 


m — l 

1 f 2 'n\ 2 


g{x*{N)) 


+ 


bJ 


UN) = (,Nf{x-{N),N)±( t 

N\NJ VI /' {x* ( At) , AT) I ^1 det f {x* (N) , N)\ ' y/N J ' 




where f is the derivative w.r.t. first coordinate and D'^f is a matrix of second derivatives 
w.r.t. all others coordinates, u{N) is some bounded function and 


KiV)| j + iffjiW) 


1 |a;B2(iV)|^ , 20r 


X 


X 


r(m) + (1 + ^ 


m—l- 






\g{x*{N))\F (^\^^,., 

Vx{F'wy 


62 + 


G'(i) mGF® \ 


{F'Wy V \/A ^ A3/2^/(2) J 




\g{x*{N))\ 

y/XF'W 


I /ArNI ^ ^ ii7(2) 1 1 _Jv1/2_F'(1) 

\ojb 2{N)\ <p ——7^62'^ H- —Ne « + ^TTTrAte 




p'(i) F'W 

where uj is a reminder as in the Theorem 2. 

Proof. We decompose I{N) into 


I{N) = F{N) + /"(At) := / g{x)e^F^F)^^ + / g{x)e^F^F)^^_ 

Jn' Jn/w 
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Then we approximate I''{N) as the integral I 2 in the proof of the previous theorem but 
with an integral over domain r2\r2' instead of fl\UN- Therefore we use flT^ and obtain 


/"(AT) I < 


vrf 

rXf)V^“ 


m+l 

2 


r(m) + 





m—1-| 




where R = mina-g^vo' \x — x*\. 

Let us define x := (x 2 , x^,..., Xm),x = (xi, x) and the curve of points of maximal values 
of / along variable Xi as x*{N) = x*{xi, N). The domain corresponding to a cross-section 
at some x\ will be denoted r2'(a;i). Since dVL is a hyperplane in the neighborhood of x*, 
hence hrst coordinate of x*{N) is hxed for all N > Nq and equal to x^. Therefore the 
integral I'{N) can be represented 


i-an' 


I'{N)= / /'(xi,iV)dxi, 


where dfl' is other then x*i boundary point of hi' on the axis of coordinate xi and 

J n(a;i) 


Since within hi' property ([H]) is valid we can apply Theorem 2 for the above integral 


m 

T'(^ m = piv/(xi,£*(xi,w),v) ( 9 {xi,x*{xi,N)) ^ uJiixi,N) \ 

\N J \^detD^f{xi,x*{xi,N),N) y/N / 


Since we integrated over the set r2(xi) the constants which are in the bound of a;/(xi, N) 
can be bounded by the constants for larger set hi listed in ([2117]). Therefore a;/(xi, N) has 
the same bound as in the Theorem 2. 

Now we integrate above expression w.r.t. Xi over the set {xl, dfl') to obtain I'{N). Since 
g divided by square root of determinant of considered Hessian matrix is bounded and has 
bounded derivative in the domain of integration we can use Theorem 1 which yields 

m 

T'iN') = p^f(^*WF)^ f _ ^(x*(A^)) _ Uj{N) '\ 

N\n) VI /' {x* {N), N) I ^det D^f{x*{N),N) N y/N )' 

where {xl,x*{xl, N)) = x*{N) and 


a;;(iV)| < |n;,(iV)| 


(|/'(x*(iV),iV)| 


+ 


|i^B2(-N)| \ 

N y’ 


Since Theorem 1 was applied on the domain which was the curve x*{N) the constants 
which are in the bound of ojbi{N) and ojb 2 {N) can be bounded by the constants for 
larger set H listed in ([2ll7|). Since the function g in the application of the Theorem 1 was 
equal to g{xi, x*{xi, N))/ det D'^f{xi, x*{xi, N), N) and 1, respectively for ujbi and 00b2 
therefore we have estimates 




+ 


Vx{F'(dy 

\9ix*{N))\ 

y/XF'W 


Ne 


+ 

_Ari/2 pKi-) 


mGF® \ G 
{F'd)y I \/A ^ A3/2F'(2) y) + 


hi) 

n 


;\rl/2 

iVe"^ + 


I / -n T-\ I F^ ^ 1 p(2) f ATlj2 f t\j1/2 

\oJB2iN)\ <p — + -^Ne-^ ^ 




y{i) ■ 


F'(i) 
























Then we combine above results with the estimate of I''{N) and obtain statement of the 


theorem. 


□ 


3 Probability application 


We consider an integral ([T]) with additionally 

f{x,N) = f{x) +e{N)a{x), 


( 11 ) 


where (t{x) is some twice differentiable function at x* and e{N) —)■ 0 as —)■ oo. 

Then we have following result 

Lemma 1. The points of maximums of the functions in m are related by 


x*{N) =x* + e{N)0{l), asN ^oo, 


( 12 ) 


Proof. The proof for the maximum of type (a) starts with use of the first order Taylor 
expansion 


Df{x*{N), N) = Df{x\ N) + D^f{xe{N), Nf{x*{N) - x*) 

Since x*{N),x* are a maximums of f{x,N),f{x) respectively and by flTT]) 

0 = e{N)Da{x*) + D‘^f{xe{N), N)^{x*{N) - x*) 
and then using fact that \x — x*\\\D‘^f {xe)~^\\~^ < \D‘^f{x 0 Y{x — a:*)| and ([3]) yields 



which is the desired result. 

For the maximum of type (b), as dVt near x* is a hyperplane, with appropriate rotation 
of coordinates if necessary, the first coordinate axis is orthogonal to that hyperplane. 
Therefore Xi will be fixed for the points of maximum x*{N),N > Nq and in the other 
coordinates x* and x*{N) are critical points. Therefore, the procedure of estimation is 
the same as for the previous type of maximum and yields the same result. □ 

For every N > Nq we define a probability measure on a measurable space (hi, 


r „Nf(x,N)j 



and corresponding random vector X{N) defined as an identity map on hi. 

3.1 Weak Law of large numbers and Central Limit Theorem 

Theorem 4 (Weak Law of large numbers). As N ^ oo the random vector X{N) con¬ 
verges in distribution to a constant vector x* and we have following estimate of its mgf 
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Proof. We prove the theorem by showing convergence of the mgf of X{N) 


^x(N ){0 = e^fPX)dx 


for 1^1 < h, for some h > 0. 

With Theorem 2 we approximate the denominator for the two cases of maximum 


(a) / 


(b) / 




1 + 




\NJ ^det D^f{x*{N),N) \ ^/N J 

IP.\ 


m — 1 
2 


i + W)f 

N\NJ \f'{x*{N),N)\^\ det f {x* (N), N)\ \ y/Nj' 


as W —)■ oo. 

Then we approximate the numerator, where the additional function in the exponent is g 
in the Theorems 


27r\ 


Jo, 


N J ^det D^f{x*{N),N) 
lf2.\ 


1 + 


Q(i) 

y/Nj' 


m—1 

2 


N\NJ \f'{x*{N),N)\^y\detD^f{x*{N),N)\ 
X f 1 + ) , as W — >■ cx). 

V \/w; 

Combining the approximations of denominators and numerators yields 
Mx{n){0 = ) as N ^oo. 

Next, we use estimate flT^ and Taylor expansion for the exponent function to obtain 

MxotK) = (l + €(Af)^^0(l)) (l + j, 

which yields the result of the theorem. □ 

We consider a random vector Y [N) for the two types of maximum 

(а) Y{N) = Vn{x* - X(iV)), 

(б) Y{N) = {N{xl - Xi(iV)), Vn{x* - X{N))f, 


X 


where X{N) = {Xi{N),X{N)) and x* = For the case (b), in order to have dfl 

at X* orthogonal to axis of Xi we might have to rotate the coordinate system with Y (N) 
dehned as inversely rotated. 

Theorem 5 (Fluctuations theorem). As N ^ oo, if in / flil) the function e{N) = 
then the distribution ofY[N) converges weakly to a Af{0, f {x*)~^) distribution for the 
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maximum of type (a). For the type (b) it converges to distribution with marginal distribu¬ 
tions exp (|/'(x*)|) and Af {0, D'^ f forYi{N) and (Y 2 {N),... ,Ym{N)) respectively. 

Furthermore, as N ^ oo we have following approximations of the mgf 


{a) Mk(k,K) = ^ + ({N)^/N0{1)\ 

(4) MymiO = + ^ + ^(]V)^/iV0(l)). 


where in the case (b) ^ = (^ 2 , • • • lim), f\x*) is a derivative w.r.t. Xi and D‘^f{x*) ^ is 
a matrix of derivatives w.r.t. all but first coordinate. 

Proof. We define /(x, N) := f{x, N) + (N) and the mgf of Y(N) is 


Le^hi^F)dx 

^Y{N){0 = 


From the strncture of Y(N) we infer that for large enongh N the properties of the fnnction 
/ are the same as of / but with the maximum at x*{N). 

First, we prove the result for the maximum of type (a). We approximate the numerator 
and denominator of the above mgf using Theorem 2 and obtain 


My{n){0 


^ ^Nf{x*{N),N)-Nf{x*{N),N) 


^detD^f{x*{N),N) r ^ 
^JdetD^f{x*{N),N) ^ 


0(1) ^ 

^/ivr 


as iV ^ 00 . 


Next, we insert the estimates flTT|l and ffT5|) from Preposition 1 


My{n){0 


^^eD^fix*)-^^+N-y^Oil)+eimVFO{l) A A + 

V ^/NJ\ ^/N)' 


We estimate the second term in the exponent exp(A^“^/^0(l) + e{N)\fNO{l)) = 1 + 
7 \r-i/ 2 ( 9 (i) _|_ e{N)\fNO{l) and simplify whole estimate by including only expressions 
least decreasing in N and we get the hrst result of the theorem. 

For the maximum of type (b) we approximate the numerator and denominator of the mgf 
using Theorem 3 


— ^Nf(x*(N),N)-Nf(x*(N),N) {N),N)\,/detD^fix*{N),N) / ^ 0(1) \ 

|/'(x*(A^),iV)| ^detD^f(x*{N),N) ^ 

where x = (x 2 ,.. .,Xm)- 

Since the boundary dQ is a hyperplane near x* and is orthogonal to axis of Xi we have 
x^(iV) = xl{N) = xl and w.r.t. other coordinates we have maximum of type (a). There¬ 
fore for the above estimate we can apply results of Preposition 1 and obtain 


My(n){0 


\nx*{N),N)\ A , Q(i) 

|/'(P(iV),iV)-6l V 


+ e(Ar)\/iVO(l) , 


Next, we use hrst order Taylor theorem for f'{x*{N),N) to obtain 

\f'{x*{N),N) - f{x*,N)\ < F^^'>{\x*{N) - x*{N)\ + |x*(iV) - x*|). 
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With use of flT2|) . and flTT|) we conclude 


f{x*{N), N) = f{x*) + as N ^oo. 

Analogical estimation procedure is for f'{x*{N),N), with the result 
f'{x*{N), N) = f{x*) + 0(l)e(iV), asN ^oo. 
With above estimates we can express 

|/'(a:*(iV),iV)| \f'{x*)\ 


\r{x*{N), iV) - 61 \r{x*) + Oil)/^/N - 61 


l + e(iV)0(l)| = 


\r{x*) -61 


(l + 6(iV)0(l)) 


0 ( 1 ) 


as N ^ oo and substituting that into last estimate of mgf yields the hnal result. 


□ 


3.2 Additional estimates 

Preposition 1. For the defined previously functions f and f with the maximum of type 
(a) and provided that e{N) = o{1/'/N) as N ^ oo we have following approximations 


i-{N) - x-{N) = 

f(x'(N), N) - f(x'(N), N) = f^eDy(x')-'(, + ^ + 
^AtlDn(x'(N),N) ^ ^ 0 ( 1 ) 
dAetDn(x'(N),N) 


(13) 

(14) 

(15) 


as N ^ oo. 


Proof. We begin with the proof of the hrst estimate. 

We use hrst order Taylor’s expansion for the vector function Df{x*{N), N) at x*{N) and 
since x*{N) and x*{N) are critical points 

D^f{xefiN),N){x*{N)-x*{N)) = ^6 

Since we have that x*{N) x* as N ^ oo, hence for large enough N the point xg^^^^N) 
is in and since ([3]) consequently 

lF(Af)-a:'(iV)|<h6L-^ 


Further, we use second order Taylor’s expansion of Df{x*{N), N) at x*{N) and with f[T7|l 
we get an upper bound 


D^f{x*{N), N)(x*{N) - x*{N)) 


1 , ,2 
< —\r{N)-x-{N)\\ 
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and substitute previous estimate in the RHS to obtain 


[x*{N) - x*{N)) - D^f{x*{N), N) 


-1 1 


Vn 


< 


_p(3)('_p'(2))-3|^|2 

2 iV 


Next we apply Taylor’s theorem at x* to f {x* [N), N) ^ and conclude with an esimate 
\\D^f{x*{N),N)-^ -D^f{x%N)-^\\ < e(iV)F(3)|o(i)|. 

Since flTTD and (/ + eA)~^ = I + 0(e), as e —>■ 0 


D^f(x\N)-^ = D^f(x*) 




I + e(N)D^f(x*) 




-1 


= D^f(x*)-" + e(N)0(l), 


Putting together above estimates we conclude 

(F(Af) - i*(Af)) - 


^ F(3)(F-P))-3|gp ^ e(iV)|0(l)5| 


2N 


\/N 


and since e(N) = o(^) 


(x*(N) - x*(N)) 


y/N 


D^f(x*)-"i 


- iv’ 


for some positive Ki. Hence we get the hrst result of the Preposition. 

We start the proof of the second result with evaluating f(x*(N), N) with use of the above 
estimate 


f(r(N),N) = flx*(N) + D^f(x*) 


^ , 0(1) 


+ 


^/]V N ’ 


N]- 


e{x*(N) + D^f(x*) 


-1 e , 0(1) 


+ 


— X 


VN\ y/N N 

and expand / using 3-rd order Taylor expansion at x* (N) and then substitute into above 

f(r(N), N) = f(x-(N), N) - -^e(^x‘(N) + D^f(xT'^ + - x* | + 




+ 




and get and upper bound as in flTT)) 

f(r(N), N) - f(x-(N), N) + -^e + ^ + I - 


-l(^D^f{xT'^ + 2j)l')D^f(x-(N),N)(^D^-f(x-)-^-L + iM) 


\/N n J 


< 


< -F(3) 
- 6 


D^f(x^ 


e , 0(1) 


+ 


y/N N 
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Then we use first order Taylor’s Theorem on D'^ f{x*{N), N) at x* and conclude with an 
estimate 


D^f{x*{N), N) = D^f{x\ N) + e{N)0{l) = D^f{x*) + e{N)0{l), 


( 16 ) 


where the second equality is due to fITT]) . 

We combine above estimates, apply Triangle and Schwarz inequalities to obtain 


f{r{N),N) - f{x*{N),N) + ^fD^fix*)-^^ 




|Q(i)l 

Ar3/2 


+ le^ii + 


1 _p(3) 


2y/N 


D^f{x*)-"^ + 


\D^f{x^ 


0 ( 1 ) 


y/N 


+ J0(1)£(1V)/W 


\/N 

D^f{x*)-^i + 


0 ( 1 ) 




+ 


Hence we have 


/(F(iV),iV) - f{x*{N),N) + —fD^fix*)-^^ 


<J^ + ^22e(lV) 


iV3/2 


^/N ’ 


where ii' 21,-^22 are some positive constants, hence we get the second result. 

Now we prove the third result of the Preposition. We start by taking hrst order Taylor 
approximation of D‘^ f [x* {N), N) at x* and with use of flT^ and the hrst result of this 
Preposition we obtain 


\D^f{r{N),N)-D^f{x*,N)\\ < 


_p(3) 

Vn 


D^fixT'^ + 


|Q(i)l' 

Vn 


Then, using property fITT]) yields 


D^f{x*{N),N) = D^f{x*) + 

Next, we divide estimate flT6|l by the above and get 


Q(i) 

\/]V ' 


D^fy{N),N) ^ ^ 0(1) 

D^f{x*{N),N) Vn' 

Further, the Taylor exapnsion of f{x) = yjx at 1 is 

/x = 1 + (c)“^'^^(x — 1), 


where c G (min(x, l),max(x, 1)). We apply this expansion with 


X = det 


/ OV(x*(iV),iV) \ 

V02/(^*(1V),1V)/ 


to obtain 


v/det02/(x*(iV),iV) ^ 

1 

det 

^02/(x*(iV),iV)\ ^ 

A/det 027 (p(Ar),iV) 

\/c 


lo2/(p(Ar),iV)/ 
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By ([8]) the determinants are bounded, therefore c is also bounded. Then with use Jacobi 
formula 


/ D^f{x*{N),N) \ 

\D^f(x*{N),N)J 


- 1 


det 



- 1 


< 


\fN' 


where > 0. Hence, combining above results leads to a statement of the Preposition. 

□ 


A Appendix 


For the the third term in the Taylor’s theorem of a real function we have 


E 


P{xe) 


dxi^dxi^dxi^ 


-{xi^ - x*^){xi^ - 0(a;i3 - x*^) 


= \D’^’^f{xe),{x-x*r^\ = 


= \{D’^^f{xe),{x-x*r^)\ < \\D^^f{xemi^-^Tn\ = \\D^^f{xe)\\\x - xT ( 17 ) 

where the last equality is due to the dehnition of tensor product. 


Lemma 2. For the integrer iV > 0, some constant a > 0 we have 


\x\ke-^-^dx < -^(aiV)-^^(r(fc + m) + (1 + 

■‘\S(Ar-i/3) 1 ( y ) 


k+m—l\ _£iJv1/3 


lxl'‘e-‘'^’‘’dx < 5^(oAf)-“t“(r(/t + m) + (1 + ‘je"”™" 

■\S(R) riy) 


(18) 


(19) 


where k is a nonnegative integer and S{N S{R) are m-dimensional spheres with the 
center at the origin and radius and R respectively. 

Proof. For the hrst integral we change the coordinate system into spherical, with x = rs^, 
and radius r = \x\ 


\x\ke-^\-\"dx = 


r^e-^^^"dsrdr. 


■‘\S(Af-i/3) 


'V-i/3 Js{r) 


As the function under the integral does not depend on the surface coordinates we can 
independently integrate over it. The surface of m-dimensional sphere of radius r is given 
by 

27rf 

^(r) = 


„m—l 


F(f) 


and 


/‘OO 


[ f / 

'V-i/3 Js{r) ^ IyI Jv-1/3 




Next, we perform a substitution t = aN{r‘^ — N which yields 

fc+m— 1 

t + dt, 


„k+ra-l^-aNr'^ ^ 


iN-y^ 


fc + m+l 

aN) 2 
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and we estimate the function u{t) = {t + with w{t) = plus some 

constant. Let us calculate the points t when u{t) = w{t). We solve this by reduction to 
quadratic equation with two roots 

_ 1 + Vl + 4aiVi/3 _ 1 - Vl + 4aiVV3 

^, h — -^-, 

Since t 2 < 0, quadratic function is convex and both functions are strictly increasing for 
t G [0, fi) we have u{t) > w(t) and opposite for t>ti. Hence, the function w{t)+w{ti) > 
u{t) for all t > 0. Therefore we have an estimate 

{t + a7V5)(fc+™-i)/2 < ^ ^ 

We substitute above into previous one and get that the integral is bounded 

fc+m— 1 

^ e-* (^t + ' dt<T{k + m) ++ 

hence putting the above results together gives the hrst result of Lemma. 

The second result can be proved analogically. □ 
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